feat: enable system prompt inject from mcp server based classifier #408
Conversation
Signed-off-by: Huamin Chen <[email protected]>
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned: 📁
|
There was a problem hiding this comment.
Pull Request Overview
This PR extends the MCP classifier functionality to support per-category system prompts and descriptions. It enables MCP servers to provide specialized system prompts for each category that the router can inject when processing queries, allowing for domain-specific LLM behavior.
Key changes:
- Added support for storing and retrieving per-category system prompts and descriptions from MCP servers
- Enhanced the
CategoryMappingandClassifierstructs with new methods to access category-specific metadata - Updated the MCP server example to demonstrate per-category system prompt usage
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/semantic-router/pkg/utils/classification/mcp_classifier_test.go | Added comprehensive tests for per-category system prompt functionality |
| src/semantic-router/pkg/utils/classification/mcp_classifier.go | Updated to store category system prompts and descriptions from MCP response |
| src/semantic-router/pkg/utils/classification/mapping.go | Added fields and methods for category system prompts and descriptions |
| src/semantic-router/pkg/utils/classification/classifier.go | Added public methods to retrieve category system prompts and descriptions |
| src/semantic-router/pkg/connectivity/mcp/api/types.go | Extended ListCategoriesResponse with system prompts and descriptions fields |
| examples/mcp-classifier-server/server.py | Enhanced to provide per-category system prompts and descriptions |
| examples/mcp-classifier-server/README.md | Updated documentation to explain per-category system prompt usage |
| config/config-mcp-classifier-example.yaml | Added extensive comments explaining the new functionality |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
3 participants
What type of PR is this?
Followup #368 to support system prompt injection for each category.
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Release Notes: Yes/No